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Abstract 

Deriving optimal designs for nonlinear models is in general challenging. One crucial 
step is to determine the number of support points needed. Current tools handle this 
on a case-by-case basis. Each combination of model, optimality criterion and objective 
requires its own proof. The celebrated de la Garza Phenomenon states that under a 
(p — l)th-degree polynomial regression model, any optimal design can be based on at 
most p design points, the minimum number of support points such that all parameters 
are estimable. Does this conclusion also hold for nonlinear models? If the answer is yes, 
it would be relatively easy to derive any optimal design, analytically or numerically. 
In this paper, a novel approach is developed to address this question. Using this new 
approach, it can be easily shown that the de la Garza phenomenon exists for many 
commonly studied nonlinear models, such as the Emax model, exponential model, 
three- and four-parameter log- linear models, Emax-PKl model, as well as many clas- 
sical polynomial regression models. The proposed approach unifies and extends many 
well-known results in the optimal design literature. It has four advantages over current 
tools: (i) it can be applied to many forms of nonlinear models; to continuous or discrete 
data; to data with homogeneous or non-homogeneous errors; (ii) it can be applied to 
any design region; (iii) it can be applied to multiple-stage optimal design; and (iv) it 
can be easily implemented. 

KEY WORDS: Locally optimal; Loewner ordering; Support points. 



1 Introduction 

The usefulness and popularity of nonlinear models have spurred a large literature on data 
analysis, but research on design selection has not kept pace. One complication in studying 
optimal designs for nonlinear models is that information matrices and optimal designs 
depend on unknown parameters. A common approach to solve this dilemma is to use 
locally optimal designs, which are based on one's best guess of the unknown parameters. 
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While a good guess may not always be available, this approach remains of value to obtain 
benchmarks for all designs (Ford, Torsney, and Wu, 1992). In fact, most available results 
are under the context of locally optimal designs. (Hereafter, the word "locally" is omitted 
for simplicity.) 

There is a vast literature on identifying good designs for a wide variety of linear models, 
but the problem is much more difficult and not nearly as well understood for nonlinear 
models. Relevant references will be provided in later sections in this paper. 

In the field of optimal designs, there exist no general approaches for identifying good 
designs for nonlinear models. There are three main reasons for this significant research 
gap. First, in nonlinear models the mathematics tends to become more difficult, which 
makes proving optimality of designs a more intricate problem. Current available tools are 
mainly based on the geometric approach by Elfving (1952) or the equivalence approach by 
Kiefer and Wolfowitz (1960). This typically means that results can only be obtained on a 
case-by-case basis. Each combination of model, optimality criterion and objective requires 
its own proof. It is not feasible to derive a general solution. Second, while linear models 
are all of the form E{y) = X/3, there is no simple canonical form for nonlinear models. 
Coupled with the first challenge, this means it is very difficult to establish unifying and 
overarching results for nonlinear models. Again, this means that individual consideration is 
typically needed for different models, different optimality criteria, and different objectives. 
Third, when considering the important practical problem of multi-stage experiments, the 
search for optimal designs becomes even more complicated because one needs to add design 
points on top of an existing design. 

Is there a practical way to overcome these challenges and derive a general approach for 
finding optimal designs for nonlinear models? One feasible strategy is to identify a subclass 
of designs with a simple format, so that one can restrict considerations to this subclass for 
any optimality problem. With a simple format, it would be relatively easy to derive an 
optimal design, analytically or numerically. 

To make this strategy meaningful, the number of support points for designs in the 
subclass should be as small as possible. By Caratheodory's theorem, we can always restrict 
our consideration to at most p(p+l)/2 design points (where p is the number of parameters). 
On the other hand, if we want all parameters to be estimable, the minimum number of 
support points should be at least p. Thus, the ideal situation is that the designs in the 
subclass have no more than p points. This reminds one of de la Garza (1954)'s result, which 
was discussed in detail by Pukelsheim (2006) under the concept of "admissibility". This 
result was named the celebrated de la Garza Phenomenon by Khuri, Mukherjee, Sinha, 
and Ghosh (2006). 

The de la Garza phenomenon can be explained as follows: Suppose we consider a 
(p — l)th-degree polynomial regression model (p parameters in total) with i.i.d. random 
errors. For any n point design where n > p, there exists a design with exactly p support 
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points such that the information matrix of the latter one is not inferior to that of the 
former one under Loewner ordering. Does this phenomenon also exist for other models? 
For nonlinear models with two parameters, Yang and Stufken (2009) provided an approach 
to identify the subclass of designs: for any design £ which does not belong to this class, 
there is a design in the class with an information matrix that dominates £ in the Loewner 
ordering. By applying this approach, they showed that many commonly studied models, 
such as logistic and probit models, are based on two design points. This result unifies and 
extends most available optimality results for binary response models. However, a limitation 
exists since it can only be applied to nonlinear models with one or two parameters. 

The purpose of this paper is to generalize Yang and Stufken (2009) to nonlinear models 
with an arbitrary number of parameters. The proposed approach makes it relatively easy 
to prove the de la Garza Phenomenon for many nonlinear models. In fact, for many com- 
monly studied nonlinear models, including the Emax model, exponential model, three- and 
four-parameter log-linear models, Emax-PKl model, as well as many classical polynomial 
regression models, it can be shown that for any given design £, there exists a design £* 
with at most p (number of parameters) points, where the information matrix under £* is 
not inferior to that of £ under Loewner ordering. Thus, when searching for an optimal 
design, one can restrict consideration to this subclass of designs, both for one-stage and 
multi-stage problems. Here, the optimal design can be for arbitrary parameter functions 
under any information matrix based-optimality criterion, including the commonly used A-, 
D-, E-, &p-, etc. criteria as well as standardized optimality criteria proposed by Dette 
(1997). Refer to Yang and Stufken (2009) for more details on the significance of these 
flexibilities. 

This paper is organized as follows. In Section 2, we introduce the strategy. Main results 
are presented in Section 3. Applications to many commonly studied nonlinear models are 
presented in Section 4. Section 5 is a short discussion. Most proofs are included in the 
Appendix. 

2 The strategy 

Suppose we have a nonlinear regression model for which at each point x the experimenter 
observes a response y. We assume that the y's are independent and follow some exponential 
distribution G with mean r/(x,6>), where 6 is p x 1 parameters vector. Typically, the 
optimal nonlinear designs are studied under approximate theory, i.e., instead of exact 
sample sizes for design points, design weights are used. An approximate design £ can be 
written as £ = Wj), i = 1, . . . , n}, where u)i > is the design weight for design point 
x,i and ^r=i w * = 1- ^ * s more convenient to rewrite £ as £ = {(ci,uji),i = 1, ...,n}, 
where q G [A, B] may depend on 9 and is one-to-one map of Xi G [U, V\. Typically, the 
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information matrix for 9 under design £ can be written as 



(2.1) 



where 



c(e, Ci ) 



/*ll(Cj 
#12 (Cj 



*12 (Ci) 
#22 (c*) 



#2p(c 4 ) 



\#lp(Ci) ^ 2 p(c i ) 



(2.2) 



pp(cj) y 



Here, P{6) is a p x p nonsingular matrix that depends on the value of 9 only. Notice that 
while 1^(9) is fixed for given 9 and £, there is flexibility on P{9) and C(9,Ci). For many 
models, we can adjust P{9) so that all ^/t's in (|2.2p are free of £ and 0. Some examples of 
(|2.1[) and (|2.2p are given in Section 4. 

Under locally optimality context, for two given designs £ = {(ci,iVi),i = l,...,n} 
and £* = {(dj,ujj),j = l,...,n}, Ig(#) < I^*(9) is equivalent to X^=i W «C(^ c «) ^ 
X^=i &jC{9, Cj) (here and elsewhere in this paper, matrix inequalities are under the Loewner 
ordering). One strategy to show 1^(9) < I^*(9) is to prove that the following equations 
hold: 



for 1 < / < t < p except for some I = t (one or more) 



3=1 



(2.3) 



(2.4) 



The development of the new tool is based on this strategy. Notice that Yang and Stufken 
(2009) used the same strategy for the p = 2 case. However, the picture for a general p 
is completely different. This is because when p = 2, the existence of £* can be based on 
the existence of one c and one Co satisfying two nonlinear equations, which can be solved 
explicitly. For an arbitrary p, it is unlikely to derive such explicit expressions for £ * since the 
existence of £* is based on the existence of multiple (approximately p(p+ l)/4) c's and cD's 
satisfying multiple (approximately p(p + l)/2) nonlinear equations. Alternative approaches 
must be employed. In the next section, some new algebra results will be provided to address 
these needs. 
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3 The approach 

In this section, we shall show that, under certain conditions, for a general nonlinear model, 
there exists a subclass of designs such that for any given design £, there exists a design £ 
in this subclass such that 1^(0) < I^*(0). We first introduce some new algebra results. 

3.1 Algebra results 

Let , . . . , be k functions defined on [A, B]. Throughout this paper, we have the 
following assumptions: 
Assumption: 

(i) \Pi , . . . , are infinity differentiable; 

(ii) fn has no zero value on [A, B\. 

Here, fn, 1 < t < k;t < I < k are defined as follows: 

f %(c), t = l,l = l,...,k 

/M(c) i(A^)'. ^.<*. *<i<*. <"> 

The structure of computations of fn can be viewed as the following lower triangular 
matrix. 




V ; ; '••/ 

The (t + l)th column is obtained from the tth. column. The Ith (I > t + 1) element of 
the (t + l)th column is the derivative of the ratio between the Tth and the t'th element of 
the tth. column. 

Lemma 1. Let ^i, . . . , ^ be k functions defined on [A,B]. Assume that fu(c) > 0, 
c G [A, B], I = 1, . . . , k. Then we have following conclusions: 

(a) when k = 2n—l. For any given A < cq < c\ < . . . < c n < B and cjj > 0, i = 1, . . . , n, 
there exist n pairs (cj,ujj), j = 0, . . . , n— 1, where c"o < c\ < c\ < C2 < . . . < c n _i < c n 
and ujj > 0, such that $3.3\) and {3.$ hold, and 113. 5\) > 0. 
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(b) when k = 2n — l. For any given A < c\ < . . . < c n < c n < B and > 0, i = 1, . . . , n, 
there exist n pairs (dj,ujj), j = 1, . . . , n, where A < c\ < c\ < C2 < . . . < c n _i < c n < 
c n < B and Cbj > 0, such that \3. 3\) and {3.4\j hold, and \3. 5\) < 0. 

(c) when k = 2n. For any given A < cq < c\ < . . . < c n < c n < B and uji > ; 
% = 1, . . . , n, there exist n + 1 pairs (c~j,LUj), j = 0, . . . , n, where A < Co < c\ < c\ < 
. . . < c n < c n < B and ujj > 0, such that i3. 3\) and hold, and \3.5\) < 0. 

(d) when k = 2n. For any given A < c\ < . . . < c n+ i < B and Ui > 0, i = 1, . . . , n + 1, 
there exist n pairs (5j,LOj), j = 1, . . . ,n, where A < c\ < c\ < . . . < c n < c n < c n +i < 
B and ujj > 0, such that i3.3\) and ft3.4\ ) hold, and \3. 5\) > 0. 

Here, 

i j 

= ^2ti j ^i(c j ),l = l,...,k-l; (3.4) 

i j 

Wi*fe(Q) - Yl VjVkicj)- (3-5) 
i j 

Yang and Stufken (2009) has proven Lemma Q] for k = 2 and 3. For arbitrary k, the 
proof is rather complicated (see Appendix). Lemma [T] requires that fi i(c) > for every 
I = l,...,k, which is very demanding. In fact, such strict conditions are not required. 
Suppose there are some fij(c) < 0, we can consider — ^(c) instead of ^/(c) depending on 
the situation, such that the corresponding fi,i{c) > for every / = 1, ... ,k. Notice that 
(|3.4p is invariant to such transformation and the sign of the inequality (|3.5p may need to 
be reversed. Thus we can have similar results as Lemma [1] with a relaxed condition. We 
are ready to present our first main theorem. 

Theorem 1. Let ^i, . . . , ^f^ be k functions defined on [A, B]. Let F(c) = nf=i fl,l( c )- ^ or 
any given N pairs (ci,u>i), where o L G [A, B] and u>i > 0, i = 1, . . . , N, there exists a set of 
pairs (5j,LUj), where dj £ [A, B] andujj > 0, such that \3. 3\) and ( fg.^l ) holds, and 113. 5|) < 0. 
Specifically, 

(a) when k = 2n — 1, N > n, and F(c) < for c E [>!,£?], there are n pairs (c~j,u>j) in 
the set and one of Cj 's is A; 

(b) when k = 2n — 1, N > n, and F(c) > for c G [^4, B\, there are n pairs (cj,u>j) in 
the set and one of Cj 's is B; 

(c) when k = 2n, N > n, and F(c) > for c £ [A,B], there are n + 1 pairs (5j,LUj) in 
the set and two of Cj 's are A and B; 
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(d) when k = In, N > n + 1, and F(c) < for c 6 [^4, -B], there are n pairs (cj,Cjj) in 
the set. 

Proof. The proofs for the above four cases are completely analogous. Here we will provide 
the proof of Theorem [TJ (a) . First, we prove the conclusion holds when N = n. From (|3.2p . 
it is easy to verify that if we change only one ^z(c) to — ^i(c), say, I = lo, and keep all other 
\P/(c)'s the same, then all fi i(x) will maintain their original signs with two exceptions: (i) 
fi ,l {c) and /z 0+ i 5 z 0+ i(c) reverse the sign when Iq < k or (ii) fk,k{c) reverse the sign when 
l = k. Among all fa, I = l,...,k, suppose a of them are negative, say fi^,. . . , fi at i a . 
Here, l\ < . . . <l a and a must be an odd number. 

When l 2b -i < I < hb (1 < b < (a - l)/2) or / > l a , *,(c) is defined as -#j(c). 
Otherwise, ^i(c) = ^i(c). We can verify that the corresponding //,/(c) > 0, 1 = 1, . . . , k by 
repeatedly using the argument for the change of signs of fu(c)'s when we change only one 
^i(c) to — \Pj(c) each time. Now let Co = A, and notice that ^(c) = — ^(c), by Lemma 
[2(a), the conclusion follows. 

Assume that Lemma Q] (a) holds for n < N < M . Now we consider TV" = M + 1. 
Following this assumption, for the M pairs (q,^), 1 < i < M, there exist n pairs (cj,ulj), 
j = 0, . . . ,n — 1, where Co = A, such that 

Af n-1 

i=l j=0 
A/ n-1 

5^w»*i(ci) = 5^57 i *{(c^),Z = l,...,/c- 1, (3.6) 

i=l i=0 

M n-1 
t=l j=0 

Consider the n — 1 pairs (cj,Uj), j = l,...,n — 1 and (cm+i,wm+i)' Apply (a) when 
N = n, there exist n pairs (c,-, a),-), j = 0, . . . , ra — 1 where Co = A, such that 

n—1 n—1 

WM+1 + = ^ 

j=l j=0 

n—1 n—1 

wm+i*i(cm+i) + = X] «( e j')> ' = 1, ■ ■ ■ i * - 1, (3.7) 

j=l j=0 

n—1 n—1 

wji/+i*)c(cm+i) + y^qjj^fcCcj) < y^^j^fc(cj). 

j=l 3=0 

Combining f)3.6|) and (|3.7p . we establish Lemma Q] (a) when TV = M + 1. By mathematical 
induction, the conclusion follows. □ 
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3.2 The main tools 



We are now ready to present our main tools. 

Theorem 2. For a nonlinear regression model, suppose the information matrix can be 
written as \2. 1\) and Cj G [A, B\. Rename all distinct ^i t , 1 < I < t < p to , . . . , 
such that (i) is one of Vl/jj, 1 < I < p and (ii) there is no V&jf = Vl/fc for I < t. Let 
F( c ) = Yi^i fi,i( c ) > c e [A-^]- F° r an V given design £, there exists a design such that 
< 1^(0)- Here, £ depends on different situations. 

(a) when k is odd and F(c) < 0, £ is based on at most (k + l)/2 points including point 
A. 

(b) when k is odd and F(c) > 0, £ is based on at most (k + l)/2 points including point 
B. 

(c) when k is even and F(c) > 0, £ is based on at most k/2 + 1 points including points 
A and B. 

(d) when k is even and F(c) < 0, £ is based on at most k/2 points. 

Proof. The proof for the four cases are completely analogous. Here we provide the proof 
of Theorem [2] (a) . 

By ()2.ip and the fact that P(0) depends on 9 only, it is sufficient to show that C^{6) < 
C^(9). Let £ = {(ci,LUi),i = 1, . . . , N}. If N < n, then we can just take £ = £. If N > n, 
by (a) of Theorem^ there exist n paris (5j,ujj), j = 0, . . . , n — 1, where cq = A, such that 
f)3.3f) . (|3.4p . and (13.5p < holds. Let £ = { (6j , ojj ) , j = 0, . . . ,n — 1}. Direct computation 
shows that the diagonal elements of C^{6) — C^{9) are either or greater than 0, and the 
off-diagonal elements are all 0. Thus the conclusion follows. □ 

Remark 1. For Cases (a), (b), and (d) of Theorem^ the conclusions stay the same if 
the interval [A,B] is not finite. For Case (a), [A, B] can be replaced by [A, oo). In this 
situation, for any given design £, we can choose B = Maxi<j<ArCj and the same conclusion 
follows. Similarly, [A, B] can be replaced by (— oo, -B] in Case (b) or (—00,00) in Case (d). 

Remark 2. There are many different ways to rename all distinct ^u, \ <l<t<pto 
Vl/x> ■ ■ ■ ! ^k- Not all orders can satisfy the requirements in Theorem^ However, as long 
as there exists one order 0/^1,...,^ such that these requirements can be satisfied, the 
conclusion holds. Notice that ^ must be one of^fu, 1 < I < p. 

4 Applications 

Theorem [2] can be applied to many commonly studied statistical models. In fact, as we 
demonstrate next, for many models, any optimal design can be based on the minimum 
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number of support points, i.e., number of support points such as all parameters are es- 
timable. As we discussed earlier, this makes it much easier to study an optimal design. 
Theorem [2] works on the information matrix directly. It is very general. It can be ap- 
plied to any models, continuous or discrete data with homogeneous or non-homogeneous 
error, as long as the information matrix can be written as (|2.ip . Here, we demonstrate its 
applications for the model 

Yij = r]{xi,e) +e tj , (4.1) 

where e^-'s are i.i.d. iV(0, <r 2 ) with known a 2 , Xi £ [L,U] is the design variable and 9 is 
a p x 1 parameter vector. Most commonly studied models can be written as (|4,ip . For a 
given design £ = {(xi,uji),i = 1, . . . , N}, the corresponding information matrix for 9 can 
be written as 

i=i 

Next, we apply Theorem for some popular choices of rj(x,0). Notice that Theorem [2] is 
not limited to this model format. It can be applied to many other models. Some examples 
will be shown in Sections 14.31 and 14.41 



4.1 Models with three parameters 

Dette, Bretz, Pepelyshev, and Pinheiro (2008) studied E max , Exponential, and Log-linear 
models. These models can be written in the form of (|4.ip with 



r](x,9) 



UlX TP 

x+9 2 ^max, 



9q + 6*iexp(x/^2) Exponential, (4-3) 
#o + 6*i log(x + O2) Log-linear. 



Here, Xi £ [L,U] C (0, 00), 9\ > 0, and 9<i > 0. They showed that local MED-optimal 
designs (MED is defined as the smallest dose producing a practically relevant response) 
are either a two points design with low end point L, or a three points design with two end 
points L and U. In fact, as the following theorem shows, any optimal design can be based 
on three points including one or two end points. 



Theorem 3. Under model (L/.3p , for an arbitrary design there exists a design with 
three support points such that 1^(9) < I^*(0). Specifically, the three points include the two 
end points L and U for the E max model; the upper end point U for the exponential model; 
and the two end points L and U for the log-linear model. 
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Proof. We first consider the E max model. By some routine algebra, it can be shown that 
the information matrix can be written as in the form of (12.11) with 



Pie) 



V 



x_ 

1 




and C(#, Ci 



/l 

Ci 



ci 



4\ 

„3 



(4.4) 
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c 2 , ^3(0) = c 3 , and \I> 4 (c) = c 4 , we can 



2, /3,3 = 3, and / 4)4 = 4. By Case (c) of 



where a = l/(xj + (9 2 ). Let ^i(c) = c, * 2 (c) 
verify that the corresponding f\ i = 1, / 22 = 
Theorem [21 the conclusion follows. 

The proofs for exponential and loglinear models are similar with different -P(#) and 
C^ie). For the exponential model, 



Pie) 



(1 



V° 



-1 



and Cie,Ci 



( 1 










Ci e 2Cl 




Cie 2ci 


c 2 e 2ci 



(4.5) 



where Cj = Xj/0 2 . Let x I / i(c) = e 
c 2 e 2c , we can verify that the corresponding /^i = e c , / 2j2 
/5 5 = 2. By Case (b) of Theorem [21 the conclusion follows. 



^ 2 (c) = ce c , ^ 3 (c) = e , ^ 4 (c) = ce , and ^ 5 (c) 



1, /3,3 = 2e c , / 4 ,4 = 1, and 



For the log-linear model, 



Pie) 



(1 

o 

v° 



and C(e, Ci) 







' 1 log(Cj) Ci 

fog(Cj) fog 2 (Ci) Cjlog(Ci) 
V Ci Cjlog(Ci) c 2 



where q = l/(xj+6> 2 ). Let ^i(c) = log(c), * 2 (c) 
or c 2 , we can verify that the corresponding f\ \ - 



= c, ^(c) = clog(c), and ^(c) = 
1/c, / 2 , 2 = 1, /3,3 = 1/c, and / 4)4 



(4.6) 



log 2 (c) 
= 2/c 2 



or 4 when ^4(0) = log 2 (c) or c 2 , respectively. Apply Case (c) of Theorem [H for any design, 
we can find a design with three points including end points L and U, such that the off- 
diagonal elements are the same and diagonal elements are either the same or larger. Thus 
the conclusion follows. □ 



Remark 3. Han and Chaloner (2003) studied D- and c- optimal design under a slightly 
different exponential model where r]ix,0) = Oq + #iexp(— # 2 x). By applying the similar 
approach as used for the exponential model in Theorem^ we can show that for an arbitrary 
design there exists a design £* with three support points including low end point L such 
that I^ie) < I^*i0). This confirms and extends results in Han and Chaloner (2003) for this 
model, while Han and Chaloner (2003) showed that the D- and c-optimal designs are based 
on three points including two end points L and U. 
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Dette, Melas, and Wong (2005) studied another version of E max model, which can be 
written in the form of (|4.ip with 



where x £ [0, T], 9q > 0, 9\ > 0, and 62 7^ 0. They showed that D- and Di-optimal designs 
are based on three points including end point T. The next theorem shows that we can 
restrict ourself with three points designs for any optimal designs. 

Theorem 4. Under model fl^. 7| ), for an arbitrary design there exists a design with 
three support points such that 1^(6) < I^*(9). 

Proof. By some routine algebra, it can be shown that the information matrix can be written 
in the form of (12.11) with 



P{9) 



/l \ / (l+^F (i+ Ci ) 3 (i+ciY » 

1 k and c{e,a)= (j^r (j^T tttIS^ 

\0 - 01 log(ei) -hi Cjlogfa) c a log( Cj ) c 2 log 2 (c a ) ) 

V 00 e ° 7 \ (l+c t ) a (l+c 8 ) 4 (l+c t ) 4 / 



(4.8) 



where a = Q 1 x i 62 . Let ^i(c) = * 2 (c) = ^^r, * 3 (c) = j£§r, and * 4 (c) = ^jj, 

*5(c) = , and * 6 (c) = c ( ^ g e) ( 4 c) - We can verify that the corresponding = -(jq^pr, 

f _ q/,1 f _ 3c+l f _ 4e(3c+2) , _ 9c 3 +15c 2 +7c+l qt1 j f _ 18c 2 +15c+2 
J2,2 - d/4, / 3)3 - -3^-, / 4 ,4 - (3c+1) 2 , /5,5 - ( 3c +2) 2 C 2 ' and W ~ c ( 9c 2 + 6c +l) " 

Notice that c > 0, this implies that F(c) < 0. By Case (d) of Theorem [2J the conclusion 
follows. □ 

Remark 4. Li and Majumdar (2008) studied D -optimal design for a three-parameter lo- 
gistic model where r](x,6) = i + g lC x°p(6> 2 x) • ^ can ^ e shown that the information matrix can 
be written in the form of \4-.8\ j with c, = #iexp(#2Xi). Thus for any arbitrary design £, 
there exists a design £* with three support points such that 1^(6) < I^*(6). This confirms 
and extends Li and Majumdar (2008) 's results. 

Han and Chaloner (2003) studied D- and c-optimal designs for a model which can be 
written in the form of (14. ip with 



V (x, 0) = log(0 o + ieX p(-0 2 x))> ( 4 -9) 

where x G [L, U] C (0, 00), #0 > 0, 0\ > 0, and 62 > 0. They showed that D- and c-optimal 
designs are based on three points including end points L and U. In fact, any optimal design 
based on information matrix can be restricted to a three-point design including lower end 
point L. 
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Theorem 5. Under \4-ty > f or an U arbitrary design £, there exists a design £* with three 
support points including lower end point L such that 1^(0) < I^*(9). 



Proof. It can be shown that the information matrix can be written in the form of (|2.ip 
with 



(do 



P{9) 




I 



and C(9, q) 



i 



1+Ci 



Cj log(cj) \ 
1+Cj \ 

Cj log(Cj) 

Cj log(Cj) lQg(Cj) c 2 log 2 (ci) 

\ 1 + Ci (1+Ci) 2 (1+Ci) 2 / 

clog(c) 



1+Ci 

1 

(i+^F 



, (4.10) 



where a = |exp(-0 2Xi ). Let *i(c) = * 2 (c) = * 3 (c) = ffgpf, and * 4 (c) 

2gg>,and¥ B (c) 



c 2 log 2 (c) 



We can verify that the corresponding / 



h,2 



/3,3 = ^r, /4,4 = -2, and / 5)5 = |. Notice that c > 0, which implies that F(c) > 0. By- 
Case (b) of Theorem [2] and q = ^exp(— 6*2X1), the conclusion follows. □ 



4.2 Models with four parameters 

Fang and Hedayat (2008) studied a composed E max -PK1 model, which can be written in 
the form of (|4.ip with 



7](X,0) = 8 + 



9iD 



D + 9 2 exp(9 3 x)' 



(4.11) 



Here, D is a positive constant, x% S [0, Z7], and 6i > 0, i = 0,1,2,3. Fang and Hedayat 
(2008) showed that local D-optimal designs are based on four points including end points 
and U. The next theorem tells us that any optimal designs can be based on four points 
designs. 

Theorem 6. Under model 1 i| ), for any arbitrary design £, there exists a design £* with 
four support points such that 1^(9) < Ig*(9). 

Proof. It can be shown that the information matrix can be written in the form of (|2.1|) 
with 



P{9) 



and C(9,Ci 



(1 





\ 


-1 


1 










1 


h. 







\0 - 


|log(%) 


J 

ej 




( 1 

1 


l 


l 


Cj log(ci) \ 


l+c, 

1 


(1+Ci) 

1 


(1+c*) 2 
Ci log (Ci) 


1+Ci 

1 


(l+c s ) 2 

1 


(1+ Ci ) a (1+Ci) a 

1 Cjlog(Ci) 


(l+c*) 2 
Ci log(cj) 

V (i+c,) 2 


(l+c s ) a 

Ci log (a) 


(1+Ci) 4 (1 + Ci) 4 
Cilog(Ci) C 2 log 2 (Ci) 
(1 + Ci) 4 (1+c .)4 y 



(4.12) 



12 



where q = ^exp(0 3 Xi). Let ^i(c) = (jq^yr, *2(c 



1 



* 5 (c) 



clog(c) 

"(l+cF' 



(l+c) 2 ' 

verify that the corresponding /1 1 



(l+c) a ' 

^4, and * 8 (c) 



and * 4 (c) 



f _ 9c 3 +15c 2 +7c+l f _ 9c(3c+2) 
J5,5 — c 2(3c +2 )2 > 76,6 



4 



9c 2 +6c+l ' 77,7 

implies that F{c) < 0. By applying Case (d) of Theorem [21 the conclusion follows 



(l+c) 

/2,2 

^£r, and / 8i8 



3 f 
41 J3,3 

2 

3c 7 ' 



(1+c) 4 

c 2 log 2 (c) 
= (1+c) 4 ■ 
3c+l f _ 4c(3c+2) 
(3c+l) 2 ' 



We can 



14,4 



~ 3c^~> fa 

Notice that c > 0, which 



□ 



Dette, Bretz, Pepelyshev, and Pinheiro (2008) studied a four-parameter logistic model, 
which can be written in the form of (|4.1h with 

0i 



Tl(x,0) =6 + 



1 + exp 



(4.13) 



Here, Xi G [L, U] C (0, 00), 6>i > 0, 9 2 > 0, and 9 3 > 0. Although they did not provide 
an analytical solution, their numerical solution shows that local MED-optimal designs are 
based on four points including the end point L. In fact, any optimal design under (|4.13p 
can be based on a four-point design. Notice that the information matrix of Model (|4.13p 
can be written as (|4.12j) except that Cj = exp( 6>2 g ~ Xi ) and 



P{9) 



/l 
1 
1 

\o o' 

Immediately, we have the following theorem. 



0\ 





01/ 



-1 



(4.14) 



Theorem 7. Under model fi4.13\ ), for any arbitrary design £, there exists a design with 
four support points such that 1^(9) < I^*(9). 

Theorem [7] confirms and extends Dette, Bretz, Pepelyshev, and Pinheiro (2008)'s nu- 
merical results. 

Remark 5. Li and Majumdar (2008) studied D -optimal design for a different version of 
four-parameter logistic model where 

^ e ) = d ° + TT % ^fl V 

1 + exp(6>2 + 6> 3 x) 

It can be shown that the information matrix can be written in the form of \4-12^ with 
Ci = exp (#2 + #3X1) and 

-1 



P{9) 



fl 








\ 





1 











1 


1 

01 





v° 





02 

01 


_|3 

01/ 



(4.15) 
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Thus for an arbitrary design £ ; there exists a design with at most four support points 
such that I^{9) < I^*(9). This confirms and extends the results in Li and Majumdar (2008) 
for this model. 



4.3 Models with p + 1 parameters 

Therorem [2] can be applied to many classical polynomial regression models, de la Garza 
(1954) studied a pth-degree polynomial regression model, which can be written in the form 
of P~Tj) with 

p 

V (x,9)=e + Y,GiX i , (4.16) 
i=i 

where x 6 [—1,1]. de la Garza (1954) proved that any optimal design can be based on at 
most p + 1 points including end points —1 and 1. Here, we provide an alternative way to 
prove this result under a general design region [L, U]. 

Theorem 8 (de la Garza Phenomenon). Under model 10) , for any arbitrary design 
there exists a design ^* with p + 1 support points including end points L and U such that 



Proof. The information matrix can be written in the form of (|2.ip with P{9) 



and 



c(e,a 



(i 



■2 



4\ 

+i 



'(p+l)x(p+l) 



(4.17) 



where q = Xj. Let ^;(c) = c , I = 1, . . . ,2p. We can check that the corresponding 
fl t l = I for I = 1, . . . , 2p. By applying Case (c) of Theorem [21 we can draw the desired 
conclusion. □ 

Weighted polynomial regression is an extension of Model (|4.16p . where the error terms 
e^-'s are i.i.d iV(0, o~ 2 /\(x)) (a 2 is known). Both Karlin and Studden (1966) and Dette, 
Haines, and Imhof (1999) studied D-optimal designs under various choices of X(x) and 
design regions. Their results show that the number of support points of D-optimal designs 
is p + 1 (except Lemma 2.2 of Dette, Haines, and Imhof, 1999, which has at most p + 2 
points). By applying Therorem O we can extend their conclusions to any optimal designs. 
The results are summarized below: 

Theorem 9. Under model 16\j , where the error terms Cij '5 are i.i.d iV(0, <j^/A(x)) (o~ 2 
is known), for an arbitrary design £, there exists a design £* such that 1^(9) < I(*(9). Here, 
is defined as follows: 
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(i) £* is based on at most p + 1 points when X(x) = (1 — x) a+1 (l + x)@ +1 , x G [—1,1], 
a + 1 > 0, and (3 + 1 > 0. 

(wj is based on at mostp + 1 points including point when \(x) = exp(— x) and x > 0. 

(Hi) £* is based on at most p+1 points when X(x) = x a+1 ex.p(— x), x > 0, and a + 1 > 0. 

(iv) £* is 6ased on at most p + 1 points when \{x) = exp(— x 2 ). 

(v) is based on at most p + 1 points when \(x) = (1 + x 2 )~ n and p < n. 

(vi) £* is based on at most p + 2 points including either lower end point L or upper end 
point U when X(x) = (1 + x 2 )~ n and p > n. 

Proof. The information matrix can be written in the form of (|2.1|) with P(9) = ^( p +i) x (p+i) 
and 



C{0,a 



( \(ci) A(cj)c, . . . A(cj)cf \ 
A(c t )Q \{ Cl )cj ... A(c t )cf +1 



\\{ Cl )4 \{ Ci )4 +l ... X( Ci )c 2p J 



(4.18) 



where q = x%. The proofs are similar for all cases except for (ii), which can be proven with 
a similar approach as in Theorem [8] proof. Here we give the proof of Theorem [9] (i) . Let 
^(c) = - J C (1 -t) a (l+tfdt and **(c) = (1 - c) a+1 (l + cf +1 c l ~ 2 , I = 2, . . . , 2p + 2. 
Notice that ^i(c) is not one of the elements in (I4.18p . We simply choose its value here for 
computation convenience. We can check that the corresponding fi i < and fij > for 
I = 2, . . . , 2p + 2. By applying Case (d) of Theorem [2j the conclusion follows. □ 

4.4 Loglinear model with quadratic term 

Theorem [2] is not limited to the model format (|4.1|) . It can be applied to any nonlinear 
model, as long as the information matrix can be written in the form of (|2.ip . Here we 
give one such example. Wang, Myers, Smith and Ye (2006) studied D-optimal designs for 
loglinear models with a quadratic term, where 

Hi ~ Poisson(//j) and log(/ij) = 9 + 9\Xi + 9 2 x 2 . (4.19) 

Here, Xi E [AC/]. They showed that D-optimal designs are based on three points for 
some selected parameters by numerical searching. Their conclusion can be verified with 
the following theorem. 

Theorem 10. Under model ^.ISfy , for any arbitrary design £, there exists a design £* 
such that 1^(9) < I^*(6). Here, when 9 2 < 0, is based on three points; when 9 2 > 0, £* 
is based on four points including the end points L and U . 
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Proof. It can be shown that the information matrix can be written in the form of (|2.ip 
with 



P{6) = exp( 



40 2 0n - 9 



and C(0,Ci 



( 



I e sign(0 2 )cf 
c . e sign(0 2 ) C 2 
,2 sign(e 2 ) C 2 



1 







6» 2 

ysign(6> 2 )4^ sign(0 2 ) 

c . e sign(e 2 )c1 c 2 e sign(6» 2 )c2 x 
sign(0 2 )cf 3 sign(0 2 )cf 



c 3gSign(0 2 )c2 








sign (0 2 ) 2 / 



(4.20) 



c «-2 e sign(e 2 )c 2 



where a = vW^ + ^-JT^- Let *i(c) = sign(0 2 ) J C e si s n ( 92 )* 2 (ft and *,(t 
Z = 2, . . . , 6. We can verify that the corresponding (i) f\ t \ < when 02 < or /^i > 
when 02 > 0; (ii) fu > for I = 2, ... ,6. By Theorem [21 the conclusion follows. □ 

Remark 6. Notice that the first derivative of e S4 3 n ( e 2)c 2 ^ s q w /j en c _ q So we cannot 
apply Theorem 2 if c ranges from a negative to a positive number and \&i(c) = e sign ^ 2 ^ c . 
To avoid this situation, a specific ^i(c) is chosen although it is not among the functions 
in ^4-. 20 ). This is the general strategy to handle such situations. The disadvantage of this 
strategy is that it could increase the number of support points unnecessarily. 



5 Discussion 



Deriving optimal designs for nonlinear models is complicated. Currently, the main tools 
are Elfving's geometric approach and Kiefer's equivalence theorem. Although these two 
approaches have been proven to be powerful tools, the results have to be derived on a case- 
by-case basis and some optimal designs are difficult to derive. In contrast, the proposed 
approach in this paper can yield very general results. As we have illustrated in the last 
section, for many commonly studied nonlinear models, this approach gives some simple 
structures based on which any optimal design can be found. As a result, it is a relatively 
easy to find an optimal design since one only needs to consider these simple structures. 
Many practical models have a moderate number of parameters (see Sections 4.1 and 4.2). 
For those models, any optimal designs can be derived readily. At a minimum, numerical 
search is feasible with the algorithm proposed by Stufken and Yang (2009). 

The well-known Caratheodory's theorem gives p{p + l)/2 as a upper bound for the 
number of support points in optimal designs. Examples in Section 4 show that the upper 
bound can be as small as p, the minimum number of support points such that all parameters 
are estimable. Although this may not be true for arbitrary nonlinear models, the proposed 
approach can be used to improve the upper bound. On the other hand, this approach gives 



16 



an alternative way to prove the de la Garza Phenomenon with little effort. Furthermore, 
this phenomenon is extended for more general weighted polynomial regression models. 

The proposed approach offers a lot of flexibility. It can be applied to multi-stage 
design, an important feature for locally optimal design. It works for any design region. 
The conditions are mild and can be easily verified using symbolic computational software 
packages, such as Maple or Mathematica. 

While the results of this paper are already far reaching, we believe that there is potential 
to extend it further. In general, this approach can be applied to any nonlinear model as 
long as the corresponding functions are differentiable. One possible obstacle is that some 
fij may take the value 0. In this situation, the proposed approach may not be applied 
directly. One way to handle it is to introduce some new functions. For example, refer 
to the proof of Theorem [TUJ This, however, may increase the number of support points 
unnecessarily. How to handle this situation remains an open question for future research. 



6 Appendix 

The purpose of this appendix is to present the proof of Lemma [TJ Most of this is done 
by mathematical induction. We first assume that Lemma [T] holds for k < K, then we will 
show that the lemma also holds for K + 1. When k = 2 and 3, Lemma [T] has been proven 
by Yang and Stufken (2009) (Propositions A. 2 and A. 3 for k = 2; Lemmas 2 and 3 for 
k = 3). There are two sections in this appendix. In Section 6.1, we present some useful 
propositions in preparation for the main proof. In the Section 6.2, we provide the main 
proof. 



6.1 Some useful propositions 



Proposition 1. Let ^f\, . . . , ^ be k functions defined on [A, B\. Assume that fu(c) > 0, 
c <E [A, B], I = 1, . . . ,k. Then for any A < c\ < . . . < c m < B, g(^[, . . . ,*' m ,ci, . . . ,c m ) > 
0, where m < k and function g is defined as 



vE^d) ^(c 2 ) ... tfKcO 



(6.1) 



Proof. By the definition of fitip) i n (|3.1j) and /11(c) > for all c G [A, B], it is sufficient 



17 



to show that 



1 1 



^(ci) ^(c 2 ) 



1 1 

&L( C A IZL( V 



fml ( „ \ /ml /„. 

hi \ c ™-i) /n \y 



fml 



> 0. 



(6.2) 



Vm — Cm 

Notice that if we treat y m as a variable ranging from c m _i to B, the determinant of the 
left hand side of (|6.2p is when y m = c m -i- So it suffices to show that its derivative about 
y m is positive when y m > c m _i, i.e., 



1 1 



/ml / _ \ fml 

11 ^ 



f 



Jl 



c 2 



/ml / 
/ll W' 



m-1; 





/ 22 (2/rr 



> 0. 



(6.3) 



-1— C m _l 



For (|6.3p . apply the same argument for y m to y m ~i, the conclusion follows if we can show 
that for y m _i G (c m _ 2 ,y m ), 



1 1 

fe(ci) fc(c 2 ) ... £ 



1 



ii W 



m- 2 J 





/ 22 (y m -i) 



m-2 



m—1. 





/ 22 (y»r 

fm2(Ur, 



> 0. 



Repeat the exact same arguments for y n 
then it is sufficient to show that 



2/m-2=Cm-2 

-2 e [c m _ 3 ,y m _i] and so on, until y 2 £ [ci,y 3 ], 



/ 22 (y 2 ) 

/3 2 (y 2 ) 



/ 22 (y3) 
732(2/3) 



frraiVm) 



> 



(6.4) 



for any yl < y 2 < 2/3 < . . . < y m < B. Notice that f 22(c) > for all c G [.A, B], repeat the 
same argument as for /i,i(c), it is sufficient to show that 



/33O3) f33(Zi) 
/43(^) fvz{zi) 



f33\Zm) 
fiziZm) 

frn3 {,Zm) 



> 0. 



(6.5) 



for any A < z% < Z4 < . . . < z m < B. Repeat the same argument for 733 and so on until 
fm-i,m-i, it is sufficient to show that f mm (c) > for any c G [A, B], which is one of our 
assumptions. Thus the conclusion follows. □ 
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Proposition 2. Let . . . , be k functions defined on [A, B\. Assume that /z,z(c) > ; 
c G [A, B], I = 1, . . . , k. Then for A < a\ < b\ < a 2 < b 2 < • • • < a m < b m < B, where 
m < k, we have 



D(*(6l) - *(oi), . . . , *(bm) - *(«m)) > 0. 

//ere, 

D(*(6i)-*(oi), • • • , *(6 m ) - *(a m )) = 

*i(&i)-*i(ai) ^1(62) -*i(oa) ••• *i(6m 
*2(6i)-*2(oi) ^2(62) - ^2(02) ••• ^2(&m 



(6.6) 



*l(flm) 
*2(flm) 



* m (6l) - *m(ai) *m(6 2 ) - *m(a 2 ) ... *m(6m)-*m(a m ) 



(6.7) 



Proof. Consider the left hand side of (|6.T[) as a function of 6 m . When 6 m = a m , it is clear 
that it is 0. The conclusion is sufficient if we can show that 



*i(6i) - *i(oi) 
^ 2 (bi) - ^2(01) 

*m(&i) - * m (oi) 



*l(&m-l) - *l(Om-l) *'i(Cm) 
*2(6n»-l) - *2(Om-l) * 2 (Cm) 

* m (& m -l) - * m (a m _i) *^(c m ) 



> 



(6.8) 



for c m £ (a m , £>]. On the other hand, the determinant in (|6.8p is zero when b m —\ — 
So the conclusion is sufficient if we can show that its derivative respective to b m -\ is positive 
for b m -i G (a m -i, c m ]. Repeat this argument for b m ^ 2 , and so on until b\. Then the 

conclusion is sufficient if 



> 



for ^4 < c\ < C2 < ■ ■ ■ < c m < B. By Proposition [IJ we have our conclusion. 



(6.9) 



□ 



Corollary 1. Let $1, . . . , be k functions defined on [A, B\. Assume that fu(c) >0, c£ 
[A, B], I = 1, . . . , k. Let S m+ i = {s\, . . . , s m+ i}, where m < k, Si £ [A,B],i = l,...,m + l 
and Si < Si+i, i = 1, . . . , m. Define 

D x (* s S m+l ) = D(*(s 2 ) - *(s0, *(s 3 ) " ^2), • • • , *(s m+ i) - ¥(a m ))> (6.10) 
where function D is defined as \6. 7\ ). Then 

D 1 (%S m+ i)>0. (6.11) 
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Proposition 3. Let ^i, . . . be k = 2n functions defined on [A,B]. Assume that 
fi,i(c) > 0, c G [A, B], I = l,...,k. Suppose for some c\ , . . . , c n and cq, . . . , c n , wheve 
A < cq < c\ < c\ < C2 < C2 < ■ ■ ■ < c n < c n < B, there exist uj\, . . . , uj n and Cjq, • • • > Wn, 
such that Y2i=l u i = Sj=o &j an< ^ ^ e following k — 1 equations hold: 

n n 

^^^(q) =J2^i(cj),l = 1,... ,k- 1. (6.12) 

i=l i=0 

// at /east one of lo\, . . . ,u n and Qq, ... ,u n is positive, then all of them should be positive. 
Under this situation, 

n n 

J2^k(ci) < ^Cjj^kicj). (6.13) 

i=\ j=0 

Proof. For ease of presentation, we define Sfc(q) = {ci, . . . , c n , cq, c\, . . . , c n } — {q} (all 
points but point q). Define r = X^'=i u; i> then we have uj n = r — YH=i u i an d 



r 



S?=o ^i- We can t rea t (16.120 as a system of linear equations u>i, . . . , u) n -i and 
ujq, . . . ,tD n _i. Let T = (& , wi,u>i, . . . ,u n -u&n-i)'\ {hi+i, 024+1) = (c n ,Cj), i = 0, . . . ,n-l; 
and (&2i,a2i) = (cj,c n ), i = 1, . . . ,n- 1, then 



/ *i(6i) -*i(ai) ... *i(6 fc _i) - *i(o fc . 



-i 



r = r 



/ ^i(c n ) - *i(c n ) 



\*k-l(&l) - *fc-i(oi) . . . $ fc _i(& fc _i) - * fc _i(a fc _i)/ V^fe-l(cn) - *fc-l(cn)y 

(6.14) 



For each tDj, j = 0, . . . ,n — 1, from (|6.14|) . by basic properties of algebra, we have 

- = D(*(h) - ^(ai), . . . , g(6fc-i) - ^(Qfe-i)) 
Wj r D(*(6 1 ) - *(ai), . . . , *(6 fc _i) - *(o fc _i)) ' 



(6.15) 



where (b{, hi) = (bi, dj) for all i / 2j + l and (623+1, «2j+i) = (c n , Cn)- Function D is defined 
as flEU) . 

We first study the property of .D(\&(oi) - *(ai), . . . , ^(^-l) - *(«fe-i))- Let Q be 
the ith column of the corresponding matrix. Let C{ = (— l) i ~ 1 [Cj + Cj-fi — C23.fi], i = 
1, • • • , 2 j — 1 , 2j+2, . . . , fc— 2, Cfc_i = Cfc_i— C23+1, C23 = —C2j+C2j+z, and C23+1 = C23+1. 
Consider the new matrix 

(Ci| . . . \C2j\C2j+2\ ■ ■ ■ \Ck~l\C2j+l)- 

Then we have 

£(*(&i) - tt(oi), . . . , *(6 fc _i) - *(o fc _i)) = (-ir^D^, S k (%)), (6.16) 
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where D\ is as defined in (|6.10p . The equality in (|6.16p holds by applying some basic 
properties of a determinant and considering the fact that k is even. 

Next, we study D{^(b\) - *(ai), . . . , ^(&fe-i) - *(afc_i)). Let Q be the ith column of 
the corresponding matrix. Let d = Cj — Cj+2 when i is odd (i < k — 1) and C^-i = Ck-i] 
Ci = —Ci + Cj+2 when i is even (i < k — 2) and Cfc_2 = Consider the new matrix 

•A4 = • • • |^fe-i)- 

Then we have 



£>(*(6i) - *(oi), . . . , *(6*_i) - *(a fc _i)) = (-l) 71 - 1 !^!. (6.17) 

Notice that the (Z,i) element of matrix A4 is ^(cm+i) — $;(c m ) when i = 2m; and 
$i(c m+ i) — f j(c m ) when i = 2m + 1. We can rewrite C\ as Ci = C\\ + (7 12 , where C\\ = 
(*i(ci) - *i(ci), . . . , * fc _i(ci) - * fc _i(ci))' and C 12 = (*i(ci) - ^i(c ), . . . , * fc _i(ci) - 
* fc _l(co))'. Then \M\ = \Mi\ + |-M 2 |, where M x = {C n \C 2 \ . . . \C k - X ) and M 2 = 
(C12IC2I • • • |Cfc_i). For matrix A^i, by subtracting the 1st column from the 2nd col- 
umn, subtracting the new 2nd column from the 3rd column, subtracting the new 3rd 
column from the 4th column, so on and so forth until the last column, we have \Mi\ = 
D\(ty, 5fc(co)). As for M. 2, we can rewrite its 3rd column C3 as C3 = C31+C32, where C31 = 
(¥1(62) - *i(c 2 ), . . . , - * fc -i(c 2 )y and (7 32 = (*i(c 2 ) - *i(ci), . . . , # fc -i(c 2 ) - 

*fc-i(ci))'. So we have |_M 2 | = \M 3 \ + |A4 4 |, where A^ 3 = (C'^ | C2 1 C31 1 C4 1 • • • |Cfc-i) 
and A4 4 = (C12IC2IC32IC4I . . . \Ck-i)- By similar argument as used for .Mi, we have 
|-M 3 | = Di(*,S fc (cD). So we have |.M| = £>i(¥, 5 fc (^)) + + |JW 4 |. As 

for A4 4 , by subtracting the 3nd column from the 2nd column, we can have a matrix which 
has similar patterns as A^2- Repeating these arguments for every odd column of each 
newly generated matrix until we reach the last column, we will have 

n 

\M\ = Y J Di(^,S k (T j )). (6.18) 
3=0 

By (lfU5l) . (IfTTBD . (f6T?) . and ffTTKD . we have 

. _ Di(*,5 fc (^)) 



E^fli(*,s k (Q)) 



,j = 0,...,n-l. (6.19) 



Since -Di^, Sk(ci)) > 0, j = 0, . . . , n, it is clear that tD^, j = 0, . . . , n have the same sign as 
r. 

On the other hand, for each Ui, i = 1, . . . , n — 1, 

D(*(h) - g(5i), . . . , - ^(afc-i)) , v 

4 D(^(6i) -*(a 1 ),...,*(6 fc _i)-*(a fc _i))' l ' ; 
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where (bj,dj) = (bj,a,j) for all j ^ 2i and (p2j,azj) = (c n ,c n ). By the similar argument as 
those for u)j,j = 0, . . . , n, we can show that 

w '= r v^n n at, o /— vn t = 1 » •••»** -1. (6.21) 

Since -Di(^, Sfc(cj)) > 0,j = 1, ...,n (Corollary 1), it is clear that u>i,i = l,...,n have 
the same sign as that of r. Thus we can draw the conclusion that if at least one number 
in cji, . . . , u n and (Do, ■■■ ,u> n is positive, then all of them are positive. Next, we will show 
that (|6.13p holds. Notice that (|6.13p is equivalent to 

r(* k (c n ) - tffcCc*)) - (* fc (6i) - * fc (oi), . . . , (*fc(6 fc -i) - tf fc (a fc _i))r > 0. (6.22) 

Here, (pi, a{),i = 1, . . . , k— 1 are defined as right before (|6.14p and F is defined as in (|6.14p . 
By Theorem 13.3.8 of Harville (1997), the left hand side of (|6.22p can be written as 

g(jg(6l) - tt(fll), ■ ■ ■ , tt(fefc-i) - ^(Cn) ~ *(Cn)) , , 

D(*(6i)-*(ai),...,*(6 fc _ 1 )-*(a fc _i)) ' K ' } 

Next, we study D(*(6i)-*(oi), . . . , *(6 fc _i)-*(ofc_i), ^(c n )-^(c n )). Let C h i = 1, . . . , k 
be the ith column of the corresponding matrix. Let C% = (— l) J_1 (Cj + Cj+i — Ck), % = 
1, . . . , k — 2 and C^-i = Cfc— i — Cfc- Consider the new matrix (C\\ . . . |Cfc_i|C&). Then we 
have 

(bi) - *(ai), . . . , ¥(6 fc _i) - *(a fc _i), *(c„) - *(c n )) = (-l) 71 - 1 ^*, S fc+1 ). (6.24) 

Here, S^+i = {c\, . . . , c n , Co, • • • , c n }. Notice that Di^, Sfc + i) > (Corollary 1). Thus, 
by (|6.17p . (|6.18p . and (|6.24p . it is clear that (|6.23p is positive when r is positive. The 
conclusion follows. □ 

Proposition 0] is analogue to Proposition [3l except that it is for the case that k is odd. 
It can be proven using a similar strategy as used in derivation of Proposition [3j We omit 
the proof due to space limit. 

Proposition 4. Let , be k = 2n — 1 functions defined on [A, B\. Assume that 

fi,i( c ) > 0, c E [A, B], I = l,...,k. Suppose for some c\ , . . . , Cn and c\ , . . . , c n , where 
A < c\ < c\ < C2 < c~2 < ■ ■ ■ < c n < c n < B, there exist oj\, . . . ,uj n and oo n , such 

that Y^i=\ u i = Y^j=x &j an d following k — 1 equations hold: 

n n 

^cu^iid) =J2^l(cj),l = l,...,k-l. (6.25) 

i=l j=l 
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If at least one number in u)\, ...,u n and u)\, . . . ,ui n is positive, then all of them should be 
positive. Under this situation, 

n n 

J2^k(ci) < ^tij^kicj). (6.26) 
i=l j=l 

Corollary 2. Let . . . , be k functions defined on [A,B]. Assume that fu(c) > 0, 
c £ [A, B], I = 1, . . . , k. Let z\ < z-i < ■ ■ ■ < Zt € [A, B] and their associated r%, . . . , r t with 
Y^,=i r i = satisfy the following k — 1 equations: 

t 

Y,r^i(zi) = 0,l = l,...,k-1. (6.27) 

i=l 

Then we have 

(a) If t < k, then ri = 0, i = 1, . . . , t; 

(b) If t = k + 1 and there exists at least one nonzero r^, then either (i) r^ > 0, i is odd 
and ri < 0, i is even; or (ii) r, L < 0, i is odd and ri > 0, i is even. 

Proof. We can rewrite ()6.27|) as 

n^i(zi) = (-n)^>i{zi), l = l,...,k-l. ( 6 28 ) 

odd i even i 

Notice that J2oddi r i = E e „ e „ i (~ r i)- 

When t < k, consider the first t — 2 equations of (|6.28p . i.e., I = 1, . . . , t — 2. Suppose 
there exists at least one nonzero r,. Applying Proposition [3] when t is odd and Proposition 
H]when t is even with k = t — 1, we have 

ri%-i(zi) + Yl (-ri)*t-i(zi). ( 6 29 ) 

odd i even i 

This contradicts (|6.28j) . So conclusion (a) follows. 

When t = k + 1 and there exists at least one nonzero r,, applying Proposition [3] when 
t is odd and Proposition H] when t is even with k = t — 1, we can draw conclusion (b). □ 

6.2 Proof of Lemma 1 

We first study some basic properties of Cj's assuming Lemma [1] holds. 

Proposition 5. Assume that Lemma{l\ holds for k < K. Let (cj,ujj) 's be the solution set 
for given (ci,uji) 's, and cq or c n (if applicable). Let to™ be a sequence of bounded positive 
number for each i and (cj 1 , uj 1 ) 's be the solution set with uji 's being replaced by uf^ 1 's and 
all other values are fixed including cq or c n (if applicable). Then we have 
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(i) dj 's are unique. 

(ii) If one of the iVi sequences, say is an increasing sequence, and all other given 
values are the same (including cq or c n if applicable) , then the dj,j < i\ are increasing 
sequences and Cj,j > i\ are decreasing sequences. On the other hand, if is an 
decreasing sequence and all other given values are the same, then Cj,j < %\ are 
decreasing sequences and Cj,j > i\ are increasing sequences. 

(Hi) If uf 1 — > Ui for all i 's, then cj 1 — * Cj for all j 's. 

(iv) If — > and uj™ 0, then either lim |c™ 4 — q 2 | = or lim|c™ — Cj 3 | = 0. 

(v) Suppose that uj™ — > 0. If there exists «2 > H, such that lim cjj" > for i > %i, then 
5™ — > Cj + \ for all j > %2 — 1. If there exist i$ < i\, such that limcj™ > for i < 13, 
then cj 1 — > Cj for all j < 13. 

(vi) If lim |c^ — Cji+i| = 0, then there exists a subsequence {mi} and i\{< ji), such that 
lima;™ 1 = and limlc™ 1 — Cj + \\ = for i\ < j < j\. Similarly, if lim |c™ — Cj 2 \ = 
0, then there exists a subsequence {1112} and i2(> 32), such that lima;™ 2 = and 
lim |c™ 2 — Cj\ =0 for 32 < j < %2- 

(vii) Suppose that lv™ < when i < i\ and uj™ = u>, otherwise. If c™ — > dj 1 , for some 
ji > i\, then — * uji for all i 's and cj 1 — ► Cj for all j 's. 

(viii) Let c™ be a sequence numbers between cq and c\, and suppose that tof 1 < uji when 
i < i\ and uj" 1 = Ui otherwise. Let (cT'jQJ 1 ) be the solution set in Case (a) for 
given (ci,{J( l ~),i = 1, . . . ,n and c™. If 'c™_ 1 — ► c ra _i, then we must have c™ — > c$ and 
lo™ ->• m, i = 1, ■ ■ ■ ,n- 

Proof, (i): Suppose (cj,Qj)'s are another solution set. By Lemma HJ we have Ylj^j + 
Ej(-%) = and 

J2%Mcj) + = 0, 1 = 1, . . . ,k - 1. (6 30) 

3 j 

By case by case discussion, notice that the given Co and c n (if applicable) are still the same, 
there are at most k distinct values among Cj's and c, 's. By rewriting the distinct values as 
Z\ < Z2 < ■ ■ ■ < Zt (t < k) and merging the associated weights into r» 3 we have Ya=i r « = 
and 

t 

Y,n^l{zi) = Q,l = l,...,k-l. (6.31) 

i=l 

By (a) of Corollary [21 we have 7*j = 0, i = 1, . . . , t. This implies Qj = iuj and Cj = dj for all 
3- 
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(ii): Suppose increases to + 5, 5 > 0, and all other given values are fixed. Let Cj's 
and Qj be the corresponding solution set. By Lemma[TJ we have 5 + &j + J2j(~<^j) = 
and 



5*i (%) + Yl Uj^iicj) + ^ (-%)*/(%) = 0, 1 = 1, . . . , k - 1. 



(6.32) 



There are at most fe + 1 distinct values among Cj x , Cj's and c^'s. Notice that 5 > and Cjj 
is distinct from Cj's and Cj's, by (a) of Corollary [21 there are exactly k + 1 distinct values 
among Cj i; Cj's and i.e., they are all distinct except the given cq and c n (if applicable). 
Notice that Qj's and o)j's are positive. By (b) of Corollary [21 if we re-order the k + 1 distinct 
values, Cj's and are alternated by Cj's. Thus we have Cj < Cj,j < i\ and dj > Cj,j > i\. 
When decreases, the proof is a completely analogous. 

(iii): Since that c m 's are bounded with Cj and Cj+i, so are limc m and lime™ - . It is 
sufficient to show that limc m = hmc m = Cj for all j's. Here, we show that the conclusion 
holds for one j, say j\. The proof for other cases are complete analogy. There exists a 
subsequence of {m}, say, {mi}, such that lime™ 1 = lime™. Then we can further choose 
a subsequence from {mi}, say, {777,2}, such that lime™ 2 exists for one j ^ j\. We can 
continue this way to choose a subsequence until we have a subsequence, say, {m'}, such 
that limc m ' = limc m , limc m ' exists for all j ^ ji, and lima)™' exists for all j. Then by 
Lemma [1] and lima/"' — > o>j, we have 



Ui = lim a)™ + ^2 nm ^ 



3 

5>i*j(ci) = lim^'^KlmTc™) + ^ limwf *,(Iim^'), i = 1, . . . , fc - 1. 



(6.33) 



By the uniqueness of Cj's from (i) and (I6.33h . we must have lime™ = Cj 1 . Similarly we can 



show that hmc™ = Cj 1 . The conclusion follows. 



(iv): By similar arguments as used in (iii), we can find a subsequence {mi}, such that 
lima/" 1 = 0, lima;" 11 > 0, lima/" 1 exists for all i ^ ii, or %2, and lime" 11 and lima;" 11 exist 
for all j's. Then by Lemma [H we have 



limo; mi + limcDj 11 ) = 0, 

3 

^lim^'^tci) + ^(- lim ^ mi )* z (lime™ 1 ) = 0, 1 = 1, . . . , k - 1. 



(6.34) 



There are at most A; distinct values among c/s, i ^ ii, and limc mi 's. By similar arguments 
as used in the proof of (i), we have either limc™l 1 = Cj 2 or lime™ 1 = Cj 2 . The conclusion 
follows. 

(v): Suppose lim a; m > for i > ii- If c™ Cj + % for some j > %i — 1, say, ji, then 
there exists a subsequence {mi}, such that lima;™ 1 exists for all i'a, lime™ 1 and lima)™ 1 
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exist for all j's, lima;™' 1 = 0, lima;™' 1 > for i > ii — 1, and lime™ -1 7^ Cj 1+ \. By similar 
arguments as used in the proof of (i), we have limcj 11 = Cj+i for j > 12 — 1 including j\. 
This is a contradiction. The case when lima;™ > for i < 13 can be proven analogously. 

(vi) : Suppose hm|c™ — Cj 1+ i| = 0. By similar argument as used in (iii), we can find a 
subsequence {mi}, such that lime™ 1 = Cj 1+ \, lime™ 1 and lima;™ 1 exist for all j, lima;™ 1 
exists for all i. Then, lima;™ 1 = for some i < j\. Otherwise, suppose that lima;™ 1 > 
for i < j\. If there is one i > j\ such that lima;™ 1 = 0, then by similar arguments as used 
in the proof of (i), we must have lime™ 1 = cj 1 . This is a contradiction. Therefore we must 
have lima;™ 1 > for all i's, so lime™ 1 = c* r Here, (c*,d)*) is the corresponding solution for 
given (q , lim a;™ 1 ) . So we have c*, < Cj 1+ \. This is also a contradiction. Thus we must have 
lima;™' 1 = for some i < j%. Let %\ be the largest i. If i\ = ji, obviously the conclusion 
holds. If i\ < j\, by the definition of i\, we have lima;™' 1 > when i\ < % < j\. By (iv) 
and the fact that lime™ 1 exists for all j's, we have either limc™^ = Cj or lime™ 1 = Cj for 
H < i < ji- But we have lime™ 1 = c^+i, which implies that lime™ 1 = Cj+i for i\ < % < j±. 
The conclusion follows. The case when lim |c™ — Cj 2 \ = is completely analogous. 

(vii) : Suppose that a;™ u>i for some i, say 12, then there must exist a subsequence 
{mi}, such that lima;™ 1 , lima)™ 1 , and lime™ 1 all exist, and lima;™ 1 < uji 2 . We have 
lime™ 1 = c'j, where (c^,oOj)'s are the solution set for given (q , lim a;™ 1 ) , i = 1, ... ,ij and 
(ci,uJi), i> i\. However, lima;™ 1 < a;, for i <i\ and lima;™ 1 < Ui 2 , we must have > dj 1 
by (ii) . This is contradictory to c™ — > Cj 1 . 

(viii) : Notice that in Case (a), if (q, u>i), i = 1, . . . , n hold the same value, we can show 
that (similar to the proof of (ii)) the solution c n is an increasing function of cq as long as 
So < c\. Apply this and use a similar approach as in (vii), the conclusion follows. □ 

For the ease of presentation, we define (C, f2) = {(ci,u>i)}, where c% < c«+i and u>i > 
0; (C, fi) = {(cj,ujj)}, where Cj < Cj + \ and ujj > 0; G/(C, fi, C, Q) = Yli^i^i^i) ~ 
J2j &j^i(cj)- For given (C, f2) with appropriate cardinality, let 

(i) Sj(C, f2,co) = Cj, where Cj's are given under Case (a); 

(ii) Sj 1 {C,Vt,c n ) = Cj, where Cj's are given under Case (b); 

(iii) Sj H (C, ft, cq, c n ) = Cj, where Cj's are given under Case (c); 

(iv) Sj (C, Q) = Cj, where c^'s are given under Case (d). 

We will use mathematical induction to prove Lemma [TJ When k = 2 and 3, Lemma 
[T]has been proven by Yang and Stufken (2009). We use the following two propositions to 
prove LemmaUfor arbitrary k, i.e., (i) assume Lemma[T]holds when k < 2n — 2, then show 
that it also holds for k = 2n — 1 and (ii) assume Lemma [T] holds when k < 2n— 1, then 
show that it also holds for k = 2n. Notice that once we can show there exist such (C, Q) 
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which satisfies (|3.3|) and (|3.4p , then by either Proposition [3] or [31 the inequality in (|3.5p 
holds. To prove the existence of (C, Q), the strategy is similar for each of the four cases. 

Proposition 6. If Lemma{]\ holds for k < 2n — 2, then it will also hold for k = 2n — 1. 

Proof. If Case (a) holds, we can consider a new function set , . . . , ^ on [—B,—A]. 
Here ^(c) = — \E'j(— c) when i is odd and ^(c) = Vl/i(— c) when z is even. For the new 
function set, we can verify that the corresponding fij > 0, c G [— 2?,— A], / = 1, . . . , k. 
Let C~ = {— Ci,i = l,...,n} and Cq = — c n . Apply Case (a) to the new function set 
Vl/l, . . . ,Vl/fc with C~ and Cq , we obtain the solution set C~ = {cj ,j = 0, . . . ,re — 1}. Let 
C = {dj = —c~_-,j = 1, . . . , n}, then Case (b) follows by replacing ^ with Vl/j. So we only 
need to prove Case (a). 

In this case, (C, SI) = {(cj,o;j),i = 1, ... ,n} and Co are given. It is sufficient to show 
that there exists a solution set (C, fi) = {(cj , ujj), j = 0, . . . , n — 1} that satisfy (|3.3[) and 
([33D of Lemma [U 

Let D = {d\, . . . , dn-i, u n }, where di G (0, Wj), i = 1, . . . , n — 1. Define Q D = {uii — 
d±, . . . , uj n -i — d n -\} and C~ n = {ci, . . . , c n _i}. We are going to show that for any given 
d n -\ G (0,u; n _i), there exist di, i = 1, . . . , n — 2, and c n _i G (c n _i,c„), such that 

S^iC-^^P^Cn-i) = Sj v (C,D), (6.35) 

for j = 1, . . . , n — 1. Once we show that (16.350 holds, we can let Qj be the corresponding 
weight of Sj H (C~ n , D , cq, c n _i), j = 0, ...,ra — 1 and dj be the corresponding weight 
of Sl v {C,D), j = l,...,n- 1. Define C = {c , Sj v (C, D), j = 1, . . . , n - 1} and U = 
{uj' ,uj'j + dj,j = 1, . . . ,n — 1}, then 

Gi(C, Q, C, tt) = 0, 1 = 1, . . . , 2n - 3. (6.36) 

It can be shown that G2 n -2(C, f2, C, O) is a continuous function of d n -\. If we further show 
that G2n-2{C, C, Q) has different signs when d n -\ \ and d ri ^\ j u; n _i, then there must 
exists a d n _i G (0,u; n _i), such that G2 n -2(C, Q, C, Q) = 0. Then our conclusion follows. 

This strategy will be achieved in three steps: (i) for any given d n ~i G (0,cj„_i) and 
c n -i G (c n _i, c n ), there exists di, i = 1, . . . ,n— 2, such that (|6.35p holds for j = 1, . . . , n — 2; 
(ii) for any given d n ^\ G (0,cj n _i), there exist c n _i G (c n _i,c n ) and di, i = 1, . . . , n — 2, 
such that (16.350 holds for j = 1, . . . , n — 1; (iii) Gm-2(C, D, C, $1) has different signs when 
rfn-i I and d n -i | o; n _i. 

Step (i) can be proven by mathematical induction. We first show that for any given 
di G (0, Ui), i = 2, . . . , n— 1 and c n _i G (c n _i, c n ), there exists <2i G (0, uj\), such that (|6.35|) 
holds when j = 1. This is because when ^ | wi, we have 

S[ n (C- n , n 5 , c , c n _i) -» c 2 and S[ V {C, D) -» ^(C, £)') < c 2 , (6.37) 
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where D' = {ui, d%, ■ ■ ■ , d n -\, u> n }. This is due to (v) and (hi) of Proposition [5j respectively. 
When d% J. 0, we have 

S[ H (C- n , n 5 , co, Cn-i) -> 5f J/ (C- n , c , Sn-i) < c 2 and S{ V (C, D) -» c 2 , (6.38) 

where O' = {cji,u;2 — da,... ,ui n -i — d n -\\. This is due to (hi) and (v) of Proposition 
respectively. By (I6.37P and (|6.38p . it is clear that 

S[ H {C~ n , £l D , co, Cn-i) — S[ V (C,D) is positive when d\ f ui and negative when di I 0. 
It can be shown that S( n (C~ n ,Q D , Co, c n -i) — S{ V (C,D) is a continuous function of di. 
Thus there exists a point d\ such that (|6.35p holds when j = 1. Notice that <ii depends on 
dj's, £ = 2, ... , n — 1 and c n _i. 

Now, we assume that for any given d{ G (0, Wj), £ = p(< n — 2), . . . , n — 1, and c n _i G 
(c n _i,c n ), there exists d, L G (0, = l,...,p — 1, such that (|6.35p holds when j = 
1, . . . ,p — 1. Consider any given d, G (0, £ = p + 1, . . . , n — 1 and c n _i. By assumption, 
for any d v G (0,u p ), there exists di G (0,u>i),i = 1, . . . ,p — 1 such that (16.35h holds when 
j = 1, . . . ,p — 1. When dp J, 0, by (v) of Proposition El we have 

Sj v (tf,.D)-c iH . 1 . (6.39) 

Next, we are going to show that d{ — > 0, £ = 1, . . . ,p— 1 when d p I 0. Suppose there exists 
some £(< p), such that di 0. Let i\ be the smallest £ that satisfies this condition. If 
i\ = 1, then we have lim|Sf^(C, D) — c\\ = by (iv) of Proposition [5j This implies that 
\im\S[ H (C- n , OP, cn. cn.-i ) - ft! [ = since flQS) holds for j = 1, . . . ,p- 1. If h > 1, then 
by (iv) of Proposition \5\ we have either limj S(^ / _ 1 (C, D) — cj 1 1 = or limj (C, D) — % | = 
0. Suppose that HmlS/^C, £>) — ct x | = 0, then we have hjn|S/ i // 1 (C'~ n , fi^, c , c n _i) - 
CiJ =0 by (|6.35p again. By (vi) of Proposition [5l lim fo;;, — di) = for some £ < %\ — 1. 
By the definition of i±, we have di — ► for i < i\. This is a contradiction. So we must 
have lim lff/ (C, D) — cj 1 1 = 0, which implies that lim |5'/ i // (C~" n , Co, c n _i) — CfJ = by 
(16351) . 

By (vi) of Proposition [5l there exists a subsequence of {d p j 0} and £2 > £i such 
that lim(u;j 2 — dj 2 ) = and lim \S( n (C~ n , Q D , cq, c n _i) — Cj| =0 for £1 < £ < £2- For 
this subsequence, by (iv) of Proposition [5] and the fact that limdj 2 = u){ 2 , we have either 
liml^/^^C,^)-^! = 0orlim|5/ 2 y (C,L>)-c i2 | =0. However, \imSl^ 1 {C,D)=\\mS I i ^ 1 {C~ n ^ ,c^c n ^) = 
Cij-i. Thus we must have lim| J S'/'^(C, .D) — c« 2 | = 0, this also implies that 

By the exact same argument, there must exist £3 > £2 and a subsequence, such that 
lining — di 3 ) = and lim |5£ /7 (C~ n , Q D , c"o, c„,_i) — Cj 3 1 = 0. Repeat this argument again, 
we can find strictly increasing numbers £4, £5, . . . and each of them has the same property 
as £2 and £3. Since p is finite, one of {£2, £3, £4, • • •} must be greater than or equal to p. 
This leads to a contradiction since all di(< Ui), i > p are fixed and d p j 0. Thus we have 
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d{ — * 0, i = 1, . . . ,p — 1 when d p j 0. This implies that 

Sj, 11 (0^,^,00,^-1) -» ^(C-^O'.co.c,,.!) < c p+1 , (6.40) 

where Q' = {wi, . . . , w p , w p +i — . . . , u n -i — d n -i}. (|6,40p is due to (iii) of Proposition 
El By dSMD and (lOOj) . we have 

^(CD) - S^iC^^^Cn-x) > (6.41) 

when eZ p j 0. 

On the other hand, notice that d p 1 uj p is equivalent to to p — d v { 0. We can show that 
the inequality sign in (|6.4ip will reverse using an analogous approach as used in the case 
of dp I 0. Due to space limit, we will just give the outline of the proof here. First, we have 

J (C- n , n 5 , co, 5n_i) - cp+i. (6.42) 

Next, we are going to show that di — ► Ui, i = 1, . . . ,p — 1 when d v \ uj p . Suppose there 
exists some i(< p), such that di Wj. Let i\ be the smallest one. We can show that 
lim | SjY [C* D) — CjJ = 0. This implies that there exists a subsequence of {d p f cj p } and 
%2 > H such that limdj 2 = and lim |5/ y (C, D) — Cj 2 | = 0. Repeat this argument, we 
can find strictly increasing numbers 13,14, ■ ■ ■ and each of them has the same property as 
%%. Since p is finite, one of {i2,^3>*4, • • •} must be greater than or equal to p. This leads 
to a contradiction since all di, i > p are fixed and d p j u p > 0. Thus we have di — > uji, 
i = 1, . . . ,p— 1 when d p f u; p . This implies that 

- S'/iCD') < c p+1 , (6.43) 

where D' = . . . , w p , d p+ \, . . . , cj n }. By (|6.42j) and (|6.43p . we have 

S™(C,D) - S 1 / 1 ^,^,^,^) < (6.44) 

when d p f u p . It can be shown that Sp V (C,D) — Sp H (C~ n ,Q, D ,co,c n -i) is a continuous 
function of d p . By (|6.4ip and (|6.44p . there must exists d p , such that 

S^iCD) = S^(C-^, co, £„_!). (6.45) 

By mathematical induction, we have shown step (i). 

Now we are going to prove step (ii). Let ^d n _i = • • • ,w n -2,4-i,Wn}- Applying 
LemmafTlwhen k = 2n—2, we have c*(d n —\) 6 (c n _i, c ra ), where c*(d n _i) = S^C, O^.J. 
From step (i), we know that for any given d n _i and c n _i £ (c*(cZ n _i), c n ), there exists dj, 
i = 1, . . . , re — 2, such that (|6.35p holds for j = 1, . . . , n — 2. It can be shown that 5^ 1 (C, D) 
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is a continuous function of Cn~\. Then it is sufficient to show that Hrn5' n _ 1 (C', D) > c n _i 
when c n _i j c*(d n _i) and lim5^ 1 (C, D) < c n _i when c n _i | c n . 

Suppose that hm5,^ 1 (C, D) < c„_i when c n _i J, c*(d„_i). There exists a subsequence 
of c n _i | c*(d n _i), such that lim S^_ X {C, D) < c n -\. Since di < u>i, i = 1, . . . , n — 2, 
by (ii) of Proposition we must have S^C.D) > .^(C, ft dn _J = c*(d„_i). This 
implies that for the subsequence of c n _i J. c*((i n _i), lim S^±(C, D) = c*(c? n _i). By (vii) of 
Proposition [5] and the fact that di < u>i, i = 1, . . . ,n — 2, we have — ► Wj, i = 1, . . . , n — 2. 
Consequently, we have 

S™ 2 (C, D) - s£ 2 (C, ft d „_J < c n „! and S™ 2 {CT n , ft 27 , c , c^-i) - c n _!, (6.46) 

by (iii) and (v) of Proposition El respectively. This is a contradiction to (I6.35P when 
j = n - 2. 

Suppose that limS'^ 1 (C, D) > c n _i when c n _i j c„. There exists a subsequence of 
Cn-l T Cn, such that Hm5'^_ 1 (C, Z?) > c n -\. By the assumption that Lemma [T] holds when 
= 2n — 2, we have S^^C, D) < c n . This implies that for the subsequence of c n _i j c„, 
S^L\{C,D) — ► c n . By (vi) of Proposition [5j there exists a sub-subsequence of c n _i f c„, 
and ii(< n — 1) (notice that d n _i is fixed), such that limd^ = and lim Sj V (C, D) = Cj+i 
for ii < j < n — 1. From the proof of step (i), limd^ = means limdj = for i < i\. On 
the other hand, we have S^ 1 (C~ n , ft D , cq, c„_i) = Cj 1+ i by f)6.35|) holds for j = i\. By (vi) 
of Proposition [5j there exists a sub-sub-subsequence of c n _i j c n and i 2 (< «i ) , such that 
\\mdi 2 = LOi 2 . This is a contradiction to limdj 2 = 0. This proves step (ii). 

Now we are going to prove step (iii). By similar arguments as used in the proof of 
lim d p = in step (i), which implies limdj = for i < p, we can show that limdj = for 
i = 1, . . . , n — 1 when d n -\ { 0. Recall the definition of (C, ft) at the beginning of the 
proof, 

G 2n . 2 (C, ft, C, ft) G 2n -2(C- n , IT™, C~ n , ft~ n ) < 0. (6.47) 

Here, (C~ n , ft _n ) is the solution set of (C~ n , ft~ n ) = {(a, u>i), i = 1, . . . , n — 1} with given 
Co and Cn{= c n ) under Case (c) of Lemma [T] when k = 2n — 2. 

Similarly, we can show that limdj = Ui for i = 1, . . . , n — 1 when d n -\ f uJ n -i- Therefore, 
we have 

G 2n . 2 (c, ft, c, ft) -» G 2n _ 2 (c, ft, c', ft') > o. (6.48) 

Here, (C, ft') is the solution set of (C, ft) under Case (d) of Lemma [T] while fc = 2n — 2. 
(|6.47|) and (16. 48ft give the proof of step (iii). This completes the proof proposition [H 

□ 

Proposition 7. If LemmaUl holds when k < 2n — 1. Then it also holds when k = 2n. 
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Proof. We first prove that Case (c) holds. In this case, 

(C, fi) = {(cj, Ui), % = 1, . . . , n}, Co and c n are given. It is sufficient to show that there exists 
a solution set (C,Q) = {(cj,Uj),j = 0, ... , n} which satisfies (j3.3j) and (|3.4p of Lemma [TJ 
The proof is similar to that in Proposition [6l Here we will provide an outline of the proof. 

Define D = {di G (0, Ui),i = 1, . . . , n} and Q D = {uji — d{, i = 1, . . . , n}. We are going 
to show that for any given d n G (0, u n ), there exist di G (0, Ui), i = 1, . . . , n — 1, such that 

S/(C, ft 17 , co) = Sf [C, D, cn) (6.49) 

for j = 1, . . . ,n — 1. Once we show that (16.49P holds, we can let a)'- be the corresponding 

weight of Sj(C, £1 , cq), j = 0, . . . , n— 1 and be the corresponding weight of Sj 7 (C, L>, c„), 

j = 1, . . . ,n. Then define C = {co, Sj(C, D , co), J = 1, • • • ,n — 1, c n } and Q = {u' ,Uj + 
dj,j = 1, . . . , n — 1, Then we have 

Gj(C, D, C, O) = 0, Z = 1, . . . , 2n - 2. (6.50) 

If we further show that G2n-i(C, D, C, Q) has different signs when d n [ and d n 1 uj n , then 
there must exists a d n G (0,w n ), such that G2 n -i(C,D,C,£l) = 0. Then our conclusion 
follows. 

This strategy will be achieved in two steps: (i) for any given d n G (0,u; n ), there exist 
di, i = 1, . . . , n — 1, such that (|6.49p holds for j = 1, . . . , n — 1; and (ii) Gm-\{pi D, C, ti) 
has different signs when d n j and d n j u n . The two steps can be proven similarly as in 
steps (i) and (iii) of Proposition [6l 

Now we shall show that Case (d) holds. In this case, (C, Q,) = {(c«, ui), i = 1, . . . , ra+ 1}. 
The proof is similar to the proof of Case (a) in Proposition [6j 

Let D = {d2, • • • , d n ,u> n +i}, where di G (0, uoi), i = 2, . . . , n. Define D = {u>x,U2 — 
d 2 ,...,uj n - d n }, C~ l = {c 2 , . . . , c n+ i} and C _(n+1 ) = {c%, . . . , c n }. We are going to show 
that for any given d n G (0, u n ), there exists di, i = 2, . . . , n — 1, c%, and c n such that 

s ii {c -(n+i)^D^ n) = sl_ x (C- l ,D,c x ) (6.51) 

for j = 1, . . . , n. Once we show that (16.5ip holds, we can define C = {Sj 1 (C~( n+1 \ £l D , c n ),j = 
1, . . . , n} with appropriate f2 (similar as that of Case (a)). Then we have 

Gi(C, Q,C,n) = 0,l = l,...,2n-2. (6.52) 

If we further show that G^n-iiC, £1, C, O) has different signs when d n [ and d n f uj n , 
then there must exist a d n G (0, uj n ), such that G2 ra -i(C, C, Q) = 0. Thus our conclusion 
follows. 

This strategy will be achieved with the following three steps: (i) for any given d n G 
(0, u) n ) and c n G [c n , c n+ \), there exists c\ G {c\, c%), di, i = 2, . . . ,n — 1, such that (|6.5ip 
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holds for j = l,...,n — 1; (ii) for any given d n G (0,u; n ), there exists c n £ (c n ,c n+ i), 
ci 6 (01,02), and d,, i = 2,...,n — 1, such that f)6.51|) holds for j = l,...,n; (hi) 
(?2 n -i(C, .D, C, fi) has different signs when d ra [ and d n f uj n . 

Define c\ = 5j 7 (C~^ n+1 \ OP , c n ) for given c^, . . . , d n and c n . Thus, we have c\ € (ci, C2) 
and (16.51j) holds j = 1. 

The proof of steps (i), (ii), and (iii) are almost exactly the same as that of Case 
(a). One only needs to change the notations and make two modifications in step (ii): 
first, c*(d n ) = S I n _ 1 (C~ 1 ,Q. dn ,ci) where £l dn = {w 2 , . . . ,u n -i,d n ,u) n+ i}; second, use (viii) 
instead of (vii) of Proposition [5j This completes the proof of Proposition [71 □ 
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